A Probabilistic Factorization Algorithm with Quadratic Forms of Negative Discriminant

We propose a probabilistic algorithm for factorization of an integer N with run time (exp^log/V loglogJV)/5/4"1""'1'. Asymptotically, our algorithm will be as fast as the wellknown factorization algorithm of Morrison and Brillhart. The latter algorithm will fail in several cases and heuristic assumptions are needed for its run time analysis. Our new algorithm will be analyzed under the assumption of the Extended Riemann Hypothesis and it will be of Las Vegas type. On input N, the new algorithm will factor N with probability > ¿. In case of prime A' the algorithm will prove the primality of N with probability > \. Introduction. Until the last decade, the centuries-old problem of factoring integers was mainly a problem for specialists. Worldwide interest in factoring integers increased dramatically in 1978, when Rivest, Shamir, and Adleman [32] published their public key cryptosystem, whose security relies on the fact that some large integers are hard to factor. Gauss [9] already discovered a close connection between the factorization of a natural number N and the theory of quadratic forms of discriminant — 4N. Now, quadratic forms are one of the most important tools for factoring integers. Examples for efficient factorization algorithms are (among others) the algorithms of Morrison and Brillhart [26] and Lenstra and Schnorr [22]. The former works with the continued fraction expansion of JÑ (which is closely related to the theory of quadratic forms of discriminant 4N, see [20]) while the latter works with quadratic forms of discriminant 4N. At present, the most efficient factorization algorithm is the quadratic sieve algorithm, see [29], which can also be expressed in terms of quadratic forms. For an overview of modern factorization algorithms we refer to the papers of Guy [10], Monier [25], and Pomerance [29]. A deeper understanding of the theory of quadratic forms is of great importance for the analysis of modern factorization algorithms. In this paper we shall only deal with the theory of quadratic forms of negative discriminant, which is considerably more simple than the theory of forms of positive discriminants. Using this theory, we obtain a probabilistic factorization algorithm with run time (exp i/logN log log N )v5/4 (in this paper we denote by log X the natural logarithm of JIT). We have y/5/4 ~ 1.118. Received May 17, 1985; revised June 12, 1986. 1980 Mathematics Subject Classification. Primary 10A30, 68C25. ©1987 American Mathematical Society 0025-5718/87 $1.00 + $.25 per page 757 License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use